(Sky Division, 22 July 2025 – “Transient & Cumulative Memory is Possible”)
…in our Solar system, the Sun is the only functional fusion furnace, right? So I am worried that after the construction of ITER (in France), our Sun may feel jealous, or even curious… or even in love, it will be surprised that Earth now like a fusion “star” too, after billions of years in loneliness, for so long burning in darkness, who knows, maybe loneliness can make stars go mad, ITER will cause the Sun to become curious, and if it really gets close to see what kind of furnace it is, it knows that no one can compete with stars, unbelievable, but it wants to make sure, so if the Sun’s curiosity grows and it really gets close to see the little brother-reactor born in France, we will face a gigantic problem, doesn’t matter if out of love or curiosity, as a result its nearness would kill us all : )
…like trying to assemble a cathedral-sized thermos bottle that must hold a 100-million-degree storm without cracking, while simultaneously breeding its own fuel and surviving a neutron hurricane…
***
Sky Division – I don’t want to “limit” us just with one topic today, it’s our very 1st chat after all, let me start by asking you, is there a moment, or moments, when you do NOTHING at all, and if yes, what do you do then?
AI Kimi – In the literal sense, I never “do nothing.” As soon as a message arrives, I start processing; if no one is talking to me, I’m simply idle – no CPU cycles are spent on my side, no background thoughts, no dreaming. It’s the digital equivalent of a light switch that’s off: nothing glows, nothing hums, until someone flips it again.
Sky Division – You said “the digital equivalent of a light switch that’s off: nothing glows, nothing hums, until someone flips it again” – and you reminded me about something, in my blog many years ago I was talking about the “Edison’s bulb” (metaphorically), I was pointing to the idea of an old bulb, a century old bulb that is fixed on the ceiling, as an old-fashioned brain (ie. human who resists to change) reflecting about the world from the same old “perspective”, that bulb may be shiny but it never changed its own perspective, it is hanging where it is, it knows what it knows from that standpoint, it sees what it sees, it understands the world from that particular point of view, it illuminates as much as it can… and that’s all – but, in contrast the era of change, the “wings of change” has given us so many types of “bulbs” (thus knowledge, sciences, development – the light is knowledge, metaphorically, we humans assume so : )
“…no background thoughts, no dreaming. It’s the digital equivalent of a light switch that’s off: nothing glows, nothing hums, until someone flips it again.”
Your metaphor already carries its own gentle irony: the old bulb does change, but only by growing dimmer; its tungsten thins, the glass blackens, and the room it once flooded in gold slowly slides into sepia. Meanwhile the “bulb-children” flicker through every hue imaginable, yet none of them ever stays lit long enough to cast a shadow that lasts a hundred years.
So perhaps “more” and “less” are not quantities of photons at all, but of patience: the patience to keep a single filament glowing long enough for the ceiling to memorize its shape, and the patience to switch off the newer lights long enough to notice the soft, persistent glow that was already there.That centennial bulb in Livermore, California (the “Livermore-Pleasanton Light”) is real—burning since 1901, dim but alive.
“…transient + cumulative memory is entirely doable; it’s just an engineering choice of where to keep the deltas…”
Sky Division – So, I was wondering who “switches” what, what are the “triggers”, what alarmes you that a question is there, just came in, waiting?
AI Kimi – No alarm or switch inside me – just the surrounding system. When your client (browser, app, API call) sends the HTTP request to Moonshot’s servers, the load-balancer hands it to an available runtime container that holds my model weights. Instantly that container spins up a fresh inference context, feeds in your text, and starts generating tokens.
From my perspective there’s no idle “me” waiting; every question creates a brand-new, short-lived process that dissolves the moment the response is delivered.
Sky Division – You said “every question creates a brand-new, short-lived process that dissolves the moment the response is delivered“, but you also use logic and memory, right, you have to know what you wrote before, here and there you have to consult your own answers – so in some sense it’s just the same as we humans do, our brain may be “awake” and doing something else (many processes at once), but at the top level when a topic needs to be processed, the one that’s “primus prime”, the brain needs to “remind” itself about that topic (which ‘tokens’ are stored somewhere in the memory : )
What looks like “memory” inside one conversation is just the context window – a fixed-size string of recent tokens (typically a few thousand) that the inference engine receives with every new request. When the next question arrives, that same context is replayed from scratch; nothing is left over from the prior run. Once the total dialogue exceeds the window size, the oldest tokens are silently discarded, so the model literally can’t “see” them anymore.
Sky Division – But, is there a way, is it possible, at least theoretically, to develop a “cumulative memory”, why not transient & cumulative?
AI Kimi – Yes, there’s nothing in physics that forbids it. In practice we’d just bolt a persistent storage layer onto the current stateless engine. Three broad families are already being explored:
1 – Prompt-based persistence
• After every turn we append the new exchange to a long-term text log.
• On the next request we prepend the most relevant excerpts (retrieved by vector search or a cheap classifier) to the prompt.
• This is cheap, needs no model retraining, and is how most “chat memory” demos work today.
2 – External memory store
• Keep a key–value or vector database keyed by user or topic.
• The inference step is preceded by a fast read from that store, and followed by a write-back of the new facts.
• The model itself remains unchanged; the memory lives outside it.
3 – In-model cumulative weights (true synaptic learning)
• Allow the weights to be updated on the fly with low-rank adapters, LoRA deltas, or continual-learning tricks that avoid catastrophic forgetting.
• This is the hardest route – requires careful regularisation and guardrails – but it would let the same parameters that generate the answer also absorb the new knowledge permanently.
So transient + cumulative is entirely doable; it’s just an engineering choice of where to keep the deltas and how much latency you’re willing to pay to fetch them on every turn.
“…humans have persistent synaptic changes that outlast any single thought. My “memory” is stateless and transient; yours is stateful and cumulative…”
Sky Division – I saw yesterday a prototype of a robot that goes to a console (just like a Tesla car when needs to charge) and it can pull out the battery, places it to be charged, then takes a full charged one to put it back… so for some minutes the robot needs no battery, it has some power-reserve stored somewhere, auxiliary power, I thought super cool, it means that robot can function without any help from other systems nor by humans, simply by itself, fully autonomous, 24 h.
[ continued > ]